Improving Masked Face Recognition Using Dense Residual Unit Aided with Quadruplet Loss

All over the world, people are wearing face masks and practising social distancing to protect themselves against the Coronavirus disease (COVID-19). The need for contactless biometric systems has increased to avoid the common point of contact. Among contactless biometric systems, facial recognition systems are the most economical and effective ones. Conventional face recognition systems rely heavily upon the facial features of the eyes, nose, and mouth. But due to people wearing face masks, the important facial features of the nose and mouth get hidden under the mask, resulting in degraded performance by the facial recognition systems on masked faces. In this paper, we propose a Dense Residual Unit (DRU) aided with Quadruplet loss on top of existing facial recognition systems. This solution tries to unveil the masked faces by producing embeddings for masked faces, which are similar to embeddings of unmasked faces of the same identity but different from embeddings of different identities. We have evaluated our method using two pre-trained facial recognition models' backbones, i.e. Resnet-101 and MobileFaceNet, and upon two datasets, among them, one is a real-world dataset, i.e. MFR2, and one is a simulated masked face dataset i.e. masked version of LFW. We have achieved improvement in the performance of masked face recognition in terms of False Match Rate, False Non-Match Rate, Fisher Discriminant ratio, and Equal Error Rate. © 2023, The Author(s), under exclusive license to Springer Nature Switzerland AG.

A Benchmark on Masked Face Recognition

With the COVID-19 pandemic's emergency, using facial masks and contactless biometric systems became even more relevant to reduce the risk of contamination. Several direct and indirect problems gained relevance with the pandemic. Among them, masked face recognition (MFR) aims to recognize a person even when the person is wearing a face mask. Some state-of-the-art algorithms that work well for unmasked faces have suffered a severe performance drop when receiving masked faces as input. In this sense, the scientific community proposed approaches and competitions related to this topic. In this paper, we introduce a comparative study of four prominent solutions pipelines that use different techniques to tackle the masked face recognition problem, proposed by Huber et al. [1], Neto et al. [2], Boutros et al. [3], and Hsu et al. [4]. The performance evaluation was conducted on a real masked face database (MFR2 [5]), and using synthetic masks in three mainstream databases (LFW, AgeDB30, and CFP-FP). We report results regarding unmasked-masked (U-M) and masked-masked (M-M) face verification performance. The unmasked-unmasked (U-U) scenario was also reported as a baseline to evaluate the drop of the selected models on non-occluded face verification. We further analyze the obtained results, generating a comprehensive comparative study of the selected approaches. © 2022 IEEE.

An Algorithm for Facial Mask Area Repair Based Upon Deep Learning

Due to the impact of Corona Virus Disease 2019 (COVID-19), facial mask has become a necessary protective measure for people going out in the last two years. One's mouth and nose are covered to suppress the spread of the virus, which brings a huge challenge for face verification. Whereas some existing image inpainting methods cannot repair the covered area well, which reduces the accuracy of face verification. In this paper, an algorithm is proposed to repair the area covered by facial mask to restore the identity information for face authentication. The proposed algorithm consists of an image inpainting network and a face verification network. Among them, in image inpainting network, to begin with, two discriminators, namely global discriminator and local discriminator. Then Resnet blocks are employed in two discriminators, which is used to retain more feature information. Experimental results show that the proposed method generates fewer artifacts and receives the higher Rank-1 accuracy than other methods in discussion. © 2022 SPIE.

Triplet Decoupling Network for Masked Face Verification

Face verification has been widely applied to identity authentication in many areas. However, due to the mask information embedded into the facial feature representation, existing face verification systems generally fail to identify the faces covered by masks during the COVID-19 coronavirus epidemic period. To address this issue, we propose a new triplet decoupling network (TDN) for the challenging masked face verification. Different from existing works, our proposed TDN seeks to remove the mask information included in extracted facial features by feature decoupling, such that more discriminative facial feature representations can be obtained for masked face verification. In addition, a new triplet similarity margin loss (TSM) is designed to enlarge the margin between the intra-class similarity and the inter-class similarity of faces. Experimental results show that the proposed method significantly outperforms the other state-of-the-art methods on masked face datasets, which demonstrates the effectiveness of our proposed method. © 2021 IEEE.

Face Verification Experiments on Moodle Quiz

The Covid-19 pandemic imposed technological challenges so that education could continue during the imposed social isolation. One of the challenges of this pandemic is online assessments. This work presents experimental results of two plugins, one for facial verification and another for access reporting, both open source, available in the Moodle Learning Management System. The first plugin was tested with 31 users. The results show the feasibility of use, with an average approval rate of 76% through a questionnaire applied to 16 users in optional online activities performed in Moodle at the end of 2020 and beginning of 2021. © 2021 IEEE.

A Lightweight Face Verification Based on Adaptive Cascade Network and Triplet Loss Function

In the past few years, with the continuous breakthrough of technology in various fields, artificial intelligence has been considered as a revolutionary technology. One of the most important and useful applications of artificial intelligence is face detection. The outbreak of COVID-19 has promoted the development of the noncontact identity authentication system. Face detection is also one of the key techniques in this kind of authentication system. However, the current real-time face detection is computationally expensive which hinders the application of face recognition. To address this issue, we propose a face verification framework based on adaptive cascade network and triplet loss. The framework is simple in network architecture and has light-weighted parameters. The training network is made of three stages with an adaptive cascade network and utilizes a novel image pyramid based on scales with different sizes. We train the face verification model and complete the verification within 0.15 second for processing one image which shows the computation efficiency of our proposed framework. In addition, the experimental results also show the competitive accuracy of our proposed framework which is around 98.6%. Using dynamic semihard triplet strategy for training, our network achieves a classification accuracy of 99.2% on the dataset of Labeled Faces in the Wild. © 2022 Jianhong Lin et al.

Challenges of data collection on mturk: A human-AI joint face-matching task

During the COVID-19 pandemic, many human-subject studies have stopped in-person data collection and shifted to virtual platforms like Amazon Mechanical Turk (MTurk). This shift involves important considerations for study design and data analysis, particularly for studies involving behavioral assessment and performance with technology. We report on lessons learned from a recent study that used MTurk for a face-matching task with an open-source AI. Participants received $5 compensation for completing a 45-minute session that included questionnaires. To help address data validity issues, Qualtrics fraud-detection features (i.e., reCAPTCHA, ID-Fraud), trap-items (e.g., Respond with Often), and a modified-batch-randomization-process were employed. Participants' accumulative accuracy and response rates were also assessed. Out of 272 participants, 121 passed the data inclusion criteria. The questionnaires' reliability was within range (average 0.78) for the healthy dataset. Accumulative accuracy in the face-matching task decreased approximately halfway through the task. Subsequent data inspection revealed that almost half of the participants spent longer than 20 seconds and up to 12 minutes on a random image pair. It is possible that participants were interrupted during the study or they elected to take unscheduled breaks. Environmental factors that were easier to control during in-person laboratory studies now require built-in controls for virtual study environments. We learned that: (1) it is imperative to monitor performance measures over time for each participant;(2) the study duration may need to be kept shorter on virtual platforms compared to in-person studies;(3) an optional, planned break during the task might help prevent other unplanned breaks. © 2021 IISE Annual Conference and Expo 2021. All rights reserved.